Modeling Human Driving Behavior through Generative Adversarial Imitation Learning

Imitation learning is an approach for generating intelligent behavior when the cost function is unknown or difficult to specify. Building upon work in inverse reinforcement learning (IRL), Generative Adversarial Imitation Learning (GAIL) aims to provide effective imitation even for problems with large or continuous state and action spaces. Driver modeling is one example of a problem where the state and action spaces are continuous. Human driving behavior is characterized by non-linearity and stochasticity, and the underlying cost function is unknown. As a result, learning from human driving demonstrations is a promising approach for generating human-like driving behavior. This article describes the use of GAIL for learning-based driver modeling. Because driver modeling is inherently a multi-agent problem, where the interaction between agents needs to be modeled, this paper describes a parameter-sharing extension of GAIL called PS-GAIL to tackle multi-agent driver modeling. In addition, GAIL is domain agnostic, making it difficult to encode specific knowledge relevant to driving in the learning process. This paper describes Reward Augmented Imitation Learning (RAIL), which modifies the reward signal to provide domain-specific knowledge to the agent. Finally, human demonstrations are dependent upon latent factors that may not be captured by GAIL. This paper describes Burn-InfoGAIL, which allows for disentanglement of latent variability in demonstrations. Imitation learning experiments are performed using NGSIM, a real-world highway driving dataset. Experiments show that these modifications to GAIL can successfully model highway driving behavior, accurately replicating human demonstrations and generating realistic, emergent behavior in the traffic flow arising from the interaction between driving agents.Comment: 28 pages, 8 figures. arXiv admin note: text overlap with arXiv:1803.0104

Analysis and synthesis of allocations of authority and responsibility in novel air traffic concepts of operation

The Next Generation Air Transportation System (NextGen) in the US and the Single European Sky Air Traffic Management (ATM) Research (SESAR) program in Europe are redefining ATM, allowing for transformative new concepts of operation that may radically re-allocate authority and responsibility between air and ground. There is a need for methods that can systematically incorporate innovative allocations of authority and responsibility in the design of novel concepts of operations to enable them to meet their specified performance and safety goals. This need translates to two objectives: 1) Create the methodology and tools for analysis of allocation of authority and responsibility in novel air traffic concepts of operation, and 2) Create the methodology and tools for synthesis of allocation of authority and responsibility in novel air traffic concepts of operation. This thesis first establishes concrete definitions of capability, authority and responsibility in the context of function allocations in the design of concepts of operations. Then, it addresses the first objective by proposing a computational modeling and simulation methodology to assess allocations of authority and responsibility with respect to the performance and safety goals of the concept of operations. Subsequently, it addresses the second objective by proposing a methodology based on network modeling and optimization to systematically synthesize allocations of authority under specified allocations of responsibility to meet performance and safety goals. The proposed methodologies are demonstrated on a case study designing of allocations of authority and responsibility in aircraft merging and spacing operations during arrival. The methodologies described and demonstrated in this thesis can be used by designers of concept of operations to both analyze and synthesize allocations of authority and responsibility. Further, the results of the case study can inform the design of similar concepts of operations.M.S

Time-bounded large-scale mission planning under uncertainty for UV disinfection

The COVID-19 pandemic has motivated research on mobile robot-based disinfection methods to help contain the spread of the virus, including ultraviolet (UV) germicidal inactivation. Recent approaches have focused on formulating autonomous disinfection as a coverage problem. However, the focus so far has been on maximising coverage, rather than scaling solutions to large-scale environments or making solutions robust to environmental uncertainty. Since the intensity of UV light is strongly coupled with the distance to the target surface, localisation errors should be included in the decision making process to synthesise meaningful irradiation durations. Therefore, in this paper we solve a linked path and dosage planning problem, explicitly considering localisation uncertainty in the model. Our model is formulated as a Markov decision process (MDP) which maps localisation uncertainty to dose delivery distributions given radiation and localisation models. We solve this (MDP) over a finite horizon using prioritised value iteration to maximise dose delivery within specified time bounds. Simulation experiments performed on real-world data show successful disinfection, outperforming a rule-based baseline

Computational Simulation of Authority-Responsibility Mismatches in Air-Ground Function Allocation

Authority-responsibility mismatches are created when one agent is authorized (has authority) to perform an activity, but a different agent is responsible for its outcome. An authority-responsibility mismatch demands monitoring by the responsible agent that itself requires additional information transfer and taskload. This paper demonstrates a computational simulation methodology that identifies when mismatches will occur in complex, multi-agent aviation operations, and their implications for information transfer between agents and task demands on each agent. A case study examines 25 authority and responsibility allocations in a NextGen/SESAR scenario in a terminal area where authority and responsibility for activities involving optimal profile descents, merging and spacing can be fluidly allocated to the aircraft (pilot/flight management system) or to the ground (air traffic controller/controller decision aids and automation)